1. Field of the Invention
The present invention relates to an image processing device, an image processing method, and a capturing device, which can correct image distortion, such as distortion aberration or chromatic aberration, produced on a captured image by an optical pickup system, with respect to an image captured by a capturing element.
2. Description of the Related Art
For example, in a capturing device such as a digital still camera and so on, schemes for obtaining a still image having a reduced noise in a time direction by creating one sheet of image through addition of plural sheets of captured images, have been proposed.
For example, Japanese Unexamined Patent Application Publication No. 2001-86398 discloses a sensorless vibration correction technology for still images. That is, Japanese Unexamined Patent Application Publication No. 2001-86398 discloses a method of capturing a plurality of images with a high-speed shutter operation that does not produce vibration, overlapping the captured images having low illumination, and obtaining one sheet of a still image having high illumination, in view of a vibration effect.
This technology is to reduce noise by dispersing the noise, which is a random component, through overlapping of sequentially captured images in view of noise reduction, and is closely related to a moving image frame noise reduction (NR) method. The moving image frame NR method is related to a technology overlapping a current frame on a reference frame in real time, in which the current frame and the reference frame consistently overlap one another in a 1:1 manner.
On the other hand, in the case of a still image NR, plural reference frames are typically given with respect to one sheet of a current frame. According to the still image NR method, the influence of vibration is reduced as the speed of a shutter becomes fast, and the noise reduction effect becomes large as the number of overlapping sheets of images is increased, so that high sensitivity can be expected.
A method of seeking noise reduction through overlapping plural sheets of images (hereinafter referred to as “addition NR process”), which is disclosed in Japanese Unexamined Patent Application Publication No. 2000-86398, is performed with respect to YC data (including a illumination component (Y) and a color component (C)) after a gamma correction and so on is performed, rather than a raw image from a capturing element.
As disclosed in Japanese Unexamined Patent Application Publication No. 2006-186884 or Japanese Unexamined Patent Application Publication No. 2007-264831, image distortion occurs in the image captured by the optical pickup system having a pickup lens in the capturing device. Such an image distortion is based on a distortion aberration characteristic of the pickup lens.
For example, when a square object is captured, as shown in FIG. 42A, if a wide-angle lens is used as the pickup lens, there is image distortion in which the square is transformed into a wine barrel shape, as shown in FIG. 42B. Further, if a telescopic lens is used as the pickup lens, there is an image distortion in which the square is transformed into a bobbin shape, as shown in FIG. 42C.
Accordingly, for the captured image, it is necessary to carry out a correction processing which eliminates or reduces the image distortion (also referred to as lens distortion) based on the distortion aberration characteristics of the pickup lens.
Several methods of correcting the lens distortion of the pickup lens have been proposed hitherto, and in addition to the method of studying the configuration of the optical pickup system, an image processing method of correcting the captured image signals from the pickup element has been disclosed.
As an image processing method, as disclosed in Japanese Unexamined Patent Application Publication No. 2007-264831, for example, Tsai's method (R. Y. Tsai: A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV camera and lenses; IEEE Journal of Robotics and Automation, RA-3,4 pp. 323-344 (1987) and Naoki Asada: Technical review and view, Computer Vision (June, 1998)) or algorisms proposed by Zhengyou Zhang (Z. Zhang: “Flexible Camera Calibration by Viewing a Plane from Unknown Orientations”, Proc. Of ICCV′ 99, pp. 666-673, 1999 and Z. Zhang: “Flexible New Technique for Camera Calibration”, IEEE, PAMI 22(11), pp. 1130-1134, 2000) has been disclosed. Further, a method has been proposed for implementing the image processing method by a software processing using a computer program (Open CV Reference Manual, 63, D Reconstruction [Camera Calibration], pp. 6_-6—3).
It is not limited to the case of the addition NR processing, and in the past, the correction of the lens distortion has been generally performed in a RAW signal state or YC signal state with respect to the captured image from the pickup element prior to a desired image processing, such as addition NR processing.